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ABSTRACT : 



An arrival process {N(t), 0 t T} is to be dispatched one 
or more times in the time interval (0,T). The problem is to deter- 
mine the optimal number of dispatches K given there are n avail- 
able and to determine sequentially the epochs of dispatch 
There are two trade off costs c^ and c^, which are respectively 
the cost per unit time of a waiting customer and the cost of dis- 
patching a single unit, A general result is found which gives us 
optimal for fixed K (i.e. the K-optimal policy) under 

certain regularity conditions. This is used to obtain subopt imal 
policies for multiple dispatching of a Poisson process and single 
dispatching of a birth-death process. Applications to problems in 
transportation, repair facilities and insect-control are indicated. 

Prepared by: 



SOME FINITE HORIZON DISPATCHING PROBLEMS 



Edward A. Brill 
Naval Postgraduate School 

1 . Introduction and Background . 

In this paper, the following general model is considered: 

Let {N(t), 0 ^ t T} denote an arrival process in the time 
interval [0,T]. Available to a central dispatcher are n dis- 
patching units to be dispatched at his discretion during this time 
interval. If a unit is dispatched to the queue site at time t, 
then the queue instantly becomes either partially or totally 
diminished, the exact assumptions depending on the context of the 
problem. Introducing a lag time is also possible. Assume that 

there are two trade-off costs c and c,, where c is the cost 

w d w 

per unit time of a waiting arrival and c^ is the cost of dis- 
patching a single unit. The problem of minimizing cost (expected 
cost) is twofold: 

(1) Determine the number of units K to be dispatched 
in [0,T], 

(2) Determine the times t ,t ,...,t when these units 

1 Z K. 

should be dispatched. 

The applications of such problems appear widespread. An 
example is the problem of dispatching buses to waiting commuters 
or generally that of dispatching a server (typically expensive) 
to a waiting line. A possibly important application of the latter 
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might be to the problem of dispatching a complicated repair facility 
to sea for repair of ships with subcritical malfunctions. Still 
another possible application is to the problem of optimally dis- 
patching insecticide-spraying units to field crops during a given 
season. In this problem c^ may be interpreted as the cost per 
unit time of an insect’s damage to crops. It also appears that 
the choice of n enters here as a constraint dictated by the 
harmful side effects of insecticides. 

Naturally, if the process {N(t), 0 ^ t sC T} is determin- 
istic then the decision variables K,T-,...,t will be explicit 

1 K 

functions of c , c,, T as well as a functional of the arrival 
w a 

process. More realistically, if this process is stochastic, then 
we have the added option of determining the dispatching times 
sequentially. 

When the arrival process is deterministic with arrival 
epochs >^2 > • • • (N>n) , the problem of finding optimal 

Ti,...,Tk for each K < n reduces to finding which of 
allocations is optimal. This is so because a dispatch should 
always be made at the instant after an arrival. After this is 
done, one may then find which K from among 0,1,..., n yields 
the optimal policy. It almost goes without saying that the above 
formulation may be handled more elegantly via dynamic programming. 

We will not enter into this here, for this author cannot foresee 
the development of any qualitative insights. For a solution to a 
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related problem, where the cumulative arrival process is continuous 
and deterministic with K = n, the reader is referred to Newell [2]. 

The simplest prototype of a dispatching problem with stochastic 

♦ 

arrivals is discussed by Ross [3], and is one in which the arrival 
process is Poisson with known rate A with one available dispatch- 
ing unit. However, it is assumed that the dispatch must be used 
so that the decision variable K is excluded from the problem. 

This was then generalized to include nonhomo geneo us Poisson arrivals 
with nonincreasing intensity function A(t). The basic tool used 
in solving these problems is the "monotone case" concept of optimal 
stopping as set forth by Chow and Robbins [1] and as extended to 
continuous time processes by Ross [4]. 

In section 2, we mention briefly a simple deterministic 
case. In section 3 we derive a general theorem for obtaining a 

K-optimal policy: that is, finding T-,...,t for a given K. 

I R 

In section 4, the result of section 3 is applied to the problem of 
multiple dispatching of a Poisson process. In section 5, we intro- 
duce into consideration the decision variable K, and in section 
6 the problem of dispatching a spatially homogeneous birth-death 



process is discussed. 
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2 • A Simple Deterministic Model . 

Suppose N(t) ^ At for 0 ^ t T, and assume that a dispatching unit 

totally diminishes the queue. For fixed K, the cost c^ does 

not affect the choice of decision variables . Thus, we 

1 K 

must find ^ < . . . < T- to minimize the total waiting time 

K K“1 1 



w, 



K 



K 

= I 

j=0 



Mx.-r.+i)2/2 



where Tq 



and 



T 



and = 0. Elementary calculus yields 

K+1 

^K-j+l " jT/(K+l) , j = 0,1,..., K+1 



W* = XT2/2(K+1) 
K 



as the K-optimal values. 

Thus for fixed K, the minimum cost of this K-optimal 
policy is 



(1) ' C*(K) = XT^c /2(K+1) + Kc • 

w d 

Minimizing (1) over the unrestricted variable K yields the value 
of K as one of the integers adjacent to 

(2) -1 + (XT2c /2 c 

w a 

Call this integer This may be seen by setting the derivative 

of C*(K) equal to zero and noting that C* (K) is convex in K. 
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It is thus clear that 



J 




if 


> n 


K* = < 


M* 


if 


0 s: M* n 


1 


^ 0 


if 


M* < 0 


T* j 

J 


= 1,. 


. . ,K* 


yields the optimal policy 



It is worthwhile to note that (2) tells us that the interval 
between dispatches is inversely proportional to the square root 
of the arrival rate X. 
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3, A General Theorem for the Stochastic Model , 

In this section we consider the problem of obtaining sequen- 
tially the times of dispatch t < for a given K, i,e, 

K i 

the K-optimal policy, Assiame without loss of generality that 

c^ = 1. Also assume total dispatching. 

Consider an integrable stochastic process {N(t), 0 t T}, 

with N(t) f a,e. Let N(t) = N(t) - N(y^) where = epoch of 

most recent dispatch before time t; i,e, let i(t) = min{n:x sit} 

n 

whence y^ = Thus the process {N(t), 0 < t si T} depends 

on the dispatching policy x < , , , < x- ; in particular N(t) 

K i 

depends on {N(s), 0 si s t) and the particular sequential policy. 
Let be the a-field generated by {N(s), 0 si s si t) and let 

denote conditional expectation given F^. We shall also add 
the continuity condition 

(3) E^N(s)^ N(t) a,e, as s \ t for every t. 



Let w^(t,T) be the j-optimal expected waiting time (assumed to 
exist) of only those arrivals in [t,T] when j dispatching times 
remain. Thus, 



WQ(t,T) = E 



[N(s) - N(t) ]ds 



The following theorem, a dynamic programming extension of the 
infinitesimal look-ahead stopping rule of Ross [4], gives us the 
K-optimal policy under certain conditions. 
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ThtoA.m 1 . A^^utrn that w^(t,T) cui^ cii^eAznttable. and 
convex j^ancXconi o^ t In [0,T] j$o/i J = 0,1,..., K-1. 
Thtn tkt [i,iiqaznttat) cLUpatcking ttmeA t <t <...<t 
dz^tnzd by = inf{t:N(t) ^ --^Wj(t,T)} aon^titut^ 

a K-optimat poticy. 



Proof . Suppose we are at time t with j + 1 dispatches remain- 
ing (j = 0,1 , . . . ,K-1) . If we dispatch at t, our "loss" from 
time t under a j-optimal policy is w^(t,T), since the cost of 
dispatching is irrelevant. If instead we go on to dispatch at 
t + £ (e > 0 arbitrary) , our optimal expected loss from t is 

rt+£ 



(4) 



N(t) • £ + E 



t J 



[N(s) - N(t)]ds + w^(t+£,X) 



We now compare (4) with w^(t,T) noting that (4) ^ w^(t,T) iff 



(5) N(t) ^ £ 



-1 



[w^(t,T)-Wj (t+£,T)] 



- E 



-1 



t+e 



E^N(s)ds 



- N(t) 



Thus dispatching at t is better than dispatching at t + £ iff (5) 
holds. By (3) as well as the convexity and differentiability of 
Wj(t,T), the right-hand side of (5) X “ w^(t,T) = -wJ(t,T) as 

£ ^ 0. Thus if N(t) ^ -wj(t,T) then dispatching now (at t) com- 
pares favorably with dispatching at t + £ for any £ > 0. Not only 
is this the case, but if we go on from t and refrain from dispatch- 
ing we should note that since -wj(t,T) is nonincreasing in t and 
N(s) nondecreasing (a.e.) for s > t, our fortunes will not change. 

That is to say, N(t) -wJ(t,T) implies N(s) ^wj(s,T) for all s > t. 
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We thus find ourselves in the monotone case (Chow and Robbins [1]) 
and hence with j + 1 dispatches remaining optimal. q.e.d. 

We may extend this theorem to include certain types of partial 
dispatching. Let be i.i.d. and concentrated on the interval 

(0,1] with EZ^ = p. Here Z^ represents the proportion of units dis- 
patched at Thus we may define N(t), the number waiting at time 

t, in the intervals t. < t < (j = K+1,K,...,1) as follows: N(t) = 

j J J- ^ J. ^ 

N(t) for ^ ^ recursively N(t) = N(t) - ^ Z.N(t.) for 

i=j ^ ^ 

Tj < t ^j-1 ^ ” K,...,l. Note that we have implicitly assumed 

that Z-,...,Z is also independent of {N(t), 0 -c t T}. 
i K 

Let WQ(t,T) be the cost of waiting due to those customers 
arriving in (t,T). In the one-stage problem, if we dispatch at 
t, our expected loss from t is 



WQ(t,T) + N(t)(l-p)(T-t). 



If we go on to dispatch at t + e, the expected loss is 



N(t)*e + WQ(t+e,T) + [N (t+e) ] (1-p) (T-t-e) + E^ 



t+e 

r 

[N(s) - N(t) ]ds . 

; 

t 



The same argument as in Theorem 1 yields e inf{t: N(t) ^ 

to be 1-optimal provided WQ(t,T) is differentiable and convex. In 

fact, E inf{t: N(t) ^ -w^(t,T)} (k = 0,1,..., j-1) is j-optimal by the same 

reasoning provided w^(t,T) is differentiable and convex for 



k = 0 , . . . , j-1 . 
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A . Multiple Dispatching of a Poisson Process . 

In this section we apply the theorem of section 3 to the 

problem of finding the K-optimal policy t when 

X K 

{N(t), 0 ^ t T} is a Poisson process with known rate X* See 
Ross [3] for the 1-optimal policy. 

For this problem EN(s) = Xs, whence 



WQ(t,T) 



X(s-t)ds = X(T"t)^/2 and 



-w’(t,T) 



= X(T-t), 



Therefore, by Theorem 1, = inf{t:N(t) ^ X(T-t)} is the 

1-optimal policy. It remains to find Wj(t,T) (J = 1,...,K-1) 

with the hope they are everywhere differentiable and convex. 

We make the simple observation that Wj(t,T) depends only 

on T - t since N(t) has stationary independent increments. 

Thus we may write Wj(t,T) E Wj(0,T-t) and proceed to investigate 

w^(0,T) as a function of T, noting that Wj(0,T) is convex in 

T iff w.(t,T) is convex in t. 

J 

From Ross [3], we find that 



(6) XT2/4 - T/2 - 1/AX s: w^(0,T) XT^/A for all T ^ 0. 

Now it remains to ponder whether w^(0,T) itself is convex in T. 

Even if this were so, it would not then follow that the 
inequality in (6) would be preserved under differentiation. It 
seems prudent to forego rigor for convenience and heurism. Since 
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the approximation in (6) results only from a possible excess over 
the boundary at it would seem that when this excess is small 

relative to XT, that w^(0,T) ^ XT^/4 is a close approximation. 

In fact, for T large, XT^/4 dominates the left hand side of 

(6) • In any event we should have 

(7) -w|(t,T) X(T-t)/2, 

If we **approximate” w^(t,T) by the left hand side of (6), then 

(8) -w{(t,T) « [X(T-t) - l]/2 

Since the three stage (three dispatch) problem depends on 
which boundary is used in the two stage problem, we would do well 
to show that the difference between the upper and lower approxi- 
mations of w^Ct,!) is **small,” In fact we shall show by induc- 
tion that the differences between successive upper and lower 
approximations of w^(t,T) are for all n bounded by 1, Suppose 
that 

(9) w _(0,T) ^ XT^/2n - g -T + g’ for some n, 

n-1 n-1 n-1 ^ 

with 0 ^ ^ 1 . Thus (9) holds for n = 1,2, with g = = 0 

n-1 U u 

and = 1/2, = 1/4X, Thus the "lower approximation" to 

-w’ , (t,T) is 
n-1 

(10) X(T-t)/n - g ^ 

n-1 
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and constitutes the boundary at the n^^ stage. Since we use (10) 
to define the n stage dispatching time we must investigate 

w^(0,T). We proceed to find a lower bound for w^(0,T): 

(11) w (0,T) i E{[X(T-t )/n - ^ - 1 ]t /2 + w , (0,T-t )} 

n n n-1 n n 

Since {N(t) - Xt} is a zero mean martingale, a simple martingale 
systems theorem yields 

(12) XEt = EN(t ) EX(T-t )/n - g . + 1 

n n n n—i 

whence 

(13) Et T/(n+l) + n(l-e ,)/X(n+l) 

n n— i 

Combining (9) with (11) , (13) and much tedious algebra yields 

(14) w (0,T) ^ XT2/2(n+l) - 3 T + 3* 

n ’ n n 

where 

(15) 3^ = 3^_j^ + (l-3j^_j^)/(n+l) for n = 1,2,... 

It may be seen from (15) that { 3 } is an increasing sequence 

n 

bounded by 1. The conclusion we may draw from this is that for 
XT/n fairly "large" it will not matter a great deal which approxi- 
mations are used. Thus, an approximate n-optimal policy is given 
by 

(16) X. = inf{t:N(t) ^ X(T-t)/j} for j = l,2,...,n 

1 
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See Figure 1. This same policy is "almost** optimal under the partial 
dispatching of section 3. 

As a final comment, we mention that if {N(t), 0 s: t ^ T} 
is nonhomogeneous Poisson with continuous intensity function X(t), 



that 



WQ(t,T) 



T 

/ 

X (u) (T-u)du 
t 



whence -w^Ct,!) = X(t)(T-t). Thus if X(t)(T-t) is nonincreasing, 
E inf{t:N(t) ^ X(t)(T^t)} is 1-optimal. The problem of finding 
the K-optimal policy for K > 1 is more difficult and will not 
be pursued. 




Figure 1. The approximate K-optimal policy for Poisson 
arrivals and K = 3. 
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5 . The Decision Variable K . 

We begin this section by discussing the simple problem 
where the dispatcher has exactly one unit which he has the option 
not to dispatch if he chooses. Thus, K may be either 0 or 1. 
The assumptions of Theorem 1 are assumed throughout. 

Consider the general stochastic model of section 3. If the 
dispatcher is contemplating use of his single unit, Theorem 1 says 
that he should dispatch at time = inf{t;N(t) ^ -w^Ct,!)}. 
Suppose, however, that at time x^ he reconsiders his decision 
and compares the expected losses associated with dispatching at 
x^ and not dispatching at all in (0,T). Clearly^ he should not 
dispatch at x^ if the former is greater; that is, if 

^ c^N(t)(T-t) + • Wq(t^,T) 

or equivalently 

(17) N(t) < cy(T-t)c^ 

Evidently, should an unexpected throng arrive shortly thereafter, 
the dispatcher could again reconsider. It then follows that the 
optimal policy is: 

(i) Dispatch at = inf{t:N(t) ^ max[-VQ(t,T) ,c^/ (T-t) ] , 

(18) (ii) Do not dispatch at all if N(t) < max[-WjJ(t,T) ,c^/(T-t) ] 



for all 0 t T. 
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When {N(t), t ^ 0} is a Poisson Process with rate A, then the 
‘‘dispatching region” is shown in Figure 2. 




Figure 2: Dispatch when process enters shaded region. 

Otherwise do not dispatch. 

We now define c^(t,T) as the optimal total expected cost, 
with j available dispatches, of all arrivals in (t,T). The 
difference here from section 3, is that we have the option to use 
a subset of these j dispatches. In principle, we would proceed 
as follows: 

(i) Determine Cj^(t,T) from (18). 

(ii) For the 2 stage problem, wait until x^, as given 

( 19 ) 

by Theorem 1, and then dispatch only if 



15 



The iteration continues, but tells us nothing of how to 
compute ^ I ^t^l^ ’ This is a complex problem as it involves 
investigation of first passage times to boundaries of a very 
intractable nature. We do not enter into such a discussion here. 
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6. Dispatching a Birth-Death Process , 

Suppose {N(t), 0 ^ t T} is a spatially homogeneous 
birth-death process, with state space consisting of all integers 
both positive and negative, and with birth parameter X and death 
parameter y. Actually this model allows for a negative queue size, 
and will serve only as an approximation to the usual birth-death 
process with reflecting barrier at 0. Thus we allow the possi- 
bility of dispatching a negative queue although this is never 
optimal. The reason we do this is that for the latter process, 
the derivations of w^(t,T) are intractable by virtue of the 
impenetrable barrier at the origin. However, we should note that 
if X >> y, which in queueing parlance means that the traffic 
intensity p >> 1, then the dispatching rule should be "very 
nearly" optimal. In fact, it is this author's conjecture that the 
same rule is optimal for both problems. 

We can no longer use Theorem 1 because we are not in the 
monotone case. This is so because {N(t)} is not nonincreasing. 
However, intuition suggests that because the "drift" of the process 
is X - y >0, we ought to use the same type rule as that dictated 
by Theorem 1 . 

Consider then the rule: 



E min[T, inf{t:N(t) ^ (X-y)(T-t)}] 
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Thzon,m 2 . Von, tkt biAXk-dnaXk pn.oce^^ rntyitiontd cut tht 
beginning tkl6 auction ^ l-optimat. 

Proof . Let a be any policy. We shall show that is good as 

a and hence optimal. If there is any t < T with 0 j; N(t)< (X-y) (T-t) 
where a tells us to dispatch, then let us modify a by choosing 
instead to dispatch at t + [ (X-y) (T-t) - N(t)]/(X-y) = t + h^. 

If N(t) < 0, we shall dispatch at T (i.e. h^ = T - t) • Call 
this modified policy and note that is as good as a since 

(X-y) (T-t)^/2 ^ N(t) • h^ + (X-y)h^/2 + (X-*y) (T-t-h^) ^/2 . 

Since N(t) may go up or down we distinguish between two cases. 

Let 

^ ^ with E > t + h^} 

Naturally P (A^) =0 if this t is such that N(t) <0. On A^, 
t + h^ = Tj^ + 6 where 6 is nonnegative and random. Since the 
expected wait from is at least 

N(t^) • 6 + (X-p)(T-t^- 6)2/2 > (X-y) (T-Tj^)2/2, 

we conclude that on A^ we should dispatch at Thus on A^, 

is as good as On if it is nonempty, we have 

t + hf < Tf so that N(t+h^) < (X-y ) (T-t-h^) . Recall, however, 
that tells us to dispatch at t + h^^. Let us modify by 



choosing to dispatch at 
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t + = t + + [ (X-u)(T-t-h^) - N(t+hj^)]/(X-p). 

If N(t+h^) < 0, let = T - t - h^. Call this policy and 

note that is as good as Such an iterative scheme will 

lead us eventually to a sequence of nested sets 



Q 

A. = > t + h- + . . . 

J 1 1 

For some i N(t+h^ + . . .+h. -) < 0 whence 
case having shown that is as good as 

proves that is as good as a. 



+ h , } • 
J 



P[A^] - 0 in which 

a . on A- ,A^ , . . . ,A. 
J 2’ ^ j 



• e • d • 
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